1. Technical Field
This application is the US national phase of international application PCT/GB00/03957 filed 16 Oct. 2000, which designated the US.
The present invention relates to a neural network component.
2. Related Art
Neural networks are biologically-inspired computation architectures for processing information. They are increasingly used to solve problems that are difficult to solve with conventional algorithmic programs running on conventional stored-program computers. These typically are pattern-matching problems such as stock market forecasting, image recognition and speech recognition. Some neural network applications are commercially significant. For example, the touch-pads of many lap-top personal computers use a neural network to improve reliability (neural systems are relatively noise-insensitive).
Neurons are generally multiple input, single output devices. The strength of the output signal from a neuron is a function of the weighted sum of that neuron's inputs and may be represented by the following equation:
      Y    i    =            f      i        ⁡          (                                    ∑            j                    ⁢                                    W              ij                        ·                          X              j                                      -                  T          i                    )      
Where Xj are inputs to the neuron (possibly from other neurons), Wij are weights applied to the inputs, the sum
          ⁢            ∑      j        ⁢                  W        ij            ·              X        j            is the activation level of the neuron (an internal measurement of the state of the neuron), Ti is the threshold of the neuron, ƒi is an activation function (this is usually non-linear), and Yi is the output of ith neuron. An output will be generated by the neuron when the activation level exceeds the threshold.
A weight associated with a given input may be positive, in which case a signal received at that input will cause the activation level to increase. A positive weight may therefore be considered to be an excitory input. In some instances a weight associated with a given input may be negative, in which case a signal received at that input will cause the activation level to decrease. A negative weight may therefore be considered to be an inhibitory input.
Connections between neurons are reinforced or weakened by adjusting the values of the weights. For example, a weight associated with a particular input of a given neuron may be increased each time a signal is received at that input. A recurring input signal (i.e. a signal received several times at the same input) will gradually increase the weight associated with that input. A signal received at that input will then cause a larger increase of the activation level of the neuron.
The activation function is usually the same for all neurons and is fixed; often a sigmoid function is used.
The activity of the ith neuron in known neural networks is limited to being a monotonic function of its inputs Xj, determined by the values of the weights Wij. This restricts the ability of known neural networks to emulate more complex, non-monotonic behaviours.